魔搭社区每周速递（11.24-11.30）_魔搭ModelScope社区-ModelScope魔搭社区

🙋魔搭ModelScope本期社区进展：

📟1361个模型：GLM-Edge系列、通义千问QwQ、Marco-o1、MagicQuill-models、Qwen2vl-Flux等；

📁29个数据集：DAVIS-Edit、modjumlu等；

🎨44个创新应用：超好用P图神器MagicQuill、logo任意贴、QwQ-32B-preview、Marco-o1等；

📄 5 篇内容：

智谱开源端侧大语言和多模态模型GLM-Edge系列！
千问团队开源会思考的QwQ模型，这模型成精了！
从 OpenAI-o1 看大模型的复杂推理能力
探索面向开放型问题的推理模型Marco-o1，阿里国际AI团队最新开源！
MagicQuill：4天斩获千颗 Star，登上Huggingface趋势榜榜首的AI P图神器

01.精选模型

GLM-Edge系列

GLM-Edge系列是智谱开源的面向端侧真实落地使用的场景下的一次尝试，由两种尺寸的大语言对话模型和多模态理解模型组成（GLM-Edge-1.5B-Chat，GLM-Edge-4B-Chat，GLM-Edge-V-2B，GLM-Edge-V-5B）。其中，1.5B / 2B模型主要面向手机、车机等平台， 4B / 5B 模型主要面向PC等平台。

模型合集：

https://modelscope.cn/collections/GLM-Edge-ff0306563d2844

示例代码：

安装源代码的transformers库

pip install git+https://github.com/huggingface/transformers.git

大语言模型推理


from modelscope import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "ZhipuAI/glm-edge-4b-chat"

tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")

message = [{"role": "user", "content": "hello!"}]

inputs = tokenizer.apply_chat_template(
    message,
    return_tensors="pt",
    add_generation_prompt=True,
    return_dict=True,
).to(model.device)

generate_kwargs = {
    "input_ids": inputs["input_ids"],
    "attention_mask": inputs["attention_mask"],
    "max_new_tokens": 128,
    "do_sample": False,
}
out = model.generate(**generate_kwargs)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

多模态模型推理


import torch
from PIL import Image
from modelscope import snapshot_download
from transformers import (
    AutoTokenizer,
    AutoImageProcessor,
    AutoModelForCausalLM,
)

url = "example.png"
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "describe this image"}]}]
image = Image.open(url)

model_dir = snapshot_download("ZhipuAI/glm-edge-v-5b")

processor = AutoImageProcessor.from_pretrained(model_dir, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_dir,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, tokenize=True, return_tensors="pt"
).to(next(model.parameters()).device)

generate_kwargs = {
    **inputs,
    "pixel_values": torch.tensor(processor(image).pixel_values).to(next(model.parameters()).device),
}
output = model.generate(**generate_kwargs, max_new_tokens=100)
print(tokenizer.decode(output[0][len(inputs["input_ids"][0]):], skip_special_tokens=True))

更多推理、微调应用详见：

智谱开源端侧大语言和多模态模型GLM-Edge系列！

QwQ-32B-Preview

QwQ-32B-Preview 是由 Qwen 团队开发的实验性研究模型，专注于增强 AI 推理能力，在数学和编程领域表现出色，具体表现如下：

GPQA：65.2%，展示了研究生水平的科学推理能力
AIME：50.0%，证明了强大的数学问题解决技能
MATH-500：90.6%，体现了在各类数学主题上的全面理解
LiveCodeBench：50.0%，验证了在实际编程场景中的出色表现

模型链接：

https://modelscope.cn/models/Qwen/QwQ-32B-Preview

示例代码：


from modelscope import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/QwQ-32B-Preview"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "How many r in strawberry."
messages = [
    {"role": "system", "content": "You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

更多体验玩法详见：

千问团队开源会思考的QwQ模型，这模型成精了！

Marco-o1

阿里国际AI团队发布了最新的Marco-o1模型，Marco-o1不仅关注具有标准答案的学科（例如代码、数学等）领域，而且更加强调开放式问题的解决方案。Marco-o1模型通过超长CoT数据微调，增强了反思和改正能力。它利用MCTS扩展解空间，输出更优结果，并定义mini-Step以提高搜索细粒度。此外，该模型在翻译任务中表现出色，首次将Inference time scaling应用于机器翻译。

模型链接：

https://modelscope.cn/models/AIDC-AI/Marco-o1

示例代码：


import torch
from typing import List, Dict, Tuple
from modelscope import AutoModelForCausalLM, AutoTokenizer

def load_model_and_tokenizer(path):
    tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(path, trust_remote_code=True).to('cuda:0')
    model.eval()
    return tokenizer, model

def generate_response(model, tokenizer,
                      input_ids, attention_mask,
                      max_new_tokens=4096):
    generated_ids = input_ids
    with torch.inference_mode():
        for _ in range(max_new_tokens):
            outputs = model(input_ids=generated_ids, attention_mask=attention_mask)
            next_token_id = torch.argmax(outputs.logits[:, -1, :], dim=-1).unsqueeze(-1)
            generated_ids = torch.cat([generated_ids, next_token_id], dim=-1)
            attention_mask = torch.cat([attention_mask, torch.ones_like(next_token_id)], dim=-1)
            new_token = tokenizer.decode(next_token_id.squeeze(), skip_special_tokens=True)
            print(new_token, end='', flush=True)
            if next_token_id.item() == tokenizer.eos_token_id:
                break
                return tokenizer.decode(generated_ids[0][input_ids.shape[-1]:], 
                                        skip_special_tokens=True)

def chat(model, tokenizer):
    history: List[Dict[str, str]] = []
    print("Enter 'q' to quit, 'c' to clear chat history.")
    while True:
        user_input = input("User: ").strip().lower()
        if user_input == 'q':
            print("Exiting chat.")
            break
            if user_input == 'c':
                print("Clearing chat history.")
                history.clear()
                continue
                if not user_input:
                    print("Input cannot be empty.")
                    continue
        history.append({"role": "user", "content": user_input})
        text = tokenizer.apply_chat_template(history, tokenize=False, add_generation_prompt=True)
        model_inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=4096).to('cuda:0')
        print('Assistant:', end=' ', flush=True)
        response = generate_response(model, tokenizer, model_inputs.input_ids, model_inputs.attention_mask)
        print()
        history.append({"role": "assistant", "content": response})

def main():
    path = "AIDC-AI/Marco-o1"
    tokenizer, model = load_model_and_tokenizer(path)
    print('Starting chat.')
    chat(model, tokenizer)

if __name__ == "__main__":
    main()

更多详情请看教程文章：

探索面向开放型问题的推理模型Marco-o1，阿里国际AI团队最新开源！